Information processing apparatus, information processing method, and program

ABSTRACT

An information processing apparatus according to the present technology includes an image obtaining unit and a display control unit. The image obtaining unit obtains a plurality of first divided images obtained by dividing a first image showing a first location along a second direction substantially perpendicular to a first direction, and a plurality of second divided images obtained by dividing a second image showing a second location along the second direction. The display control unit arranges and simultaneously displays the plurality of first divided images and the plurality of second divided images along the first direction on a display device of a user at a third location.

TECHNICAL FIELD

The present technology relates to an information processing apparatus,an information processing method, and a program that are applicable todisplay control of virtual reality (VR), augmented reality (AR), and thelike.

BACKGROUND ART

In the image processing apparatus described in Patent Literature 1, theposition and posture of a virtual object, and the position of a virtualviewpoint and the direction of a virtual line of sight of a user arestored. On the basis of the change in the position of the virtualviewpoint and the direction of the virtual line of sight of the user, animage representing a state where the virtual object is viewed from theposition of the virtual viewpoint of the user in the direction of thevirtual line of sight is displayed. This makes it possible for the userto change the view of the virtual object with a simple operation (see,e.g., paragraphs [0058] and [0063] of the specification, FIGS. 7 and 8,and the like of Patent Literature 1).

CITATION LIST Patent Literature

Patent Literature 1: Japanese Patent Application Laid-open No.2011-191899

DISCLOSURE OF INVENTION Technical Problem

As described above, there is a need for a technique capable of providinga high-quality viewing experience regarding the display control of VRand the like.

In view of the above circumstances, it is an object of the presenttechnology to provide an information processing apparatus, aninformation processing method, and a program that are capable ofproviding a high-quality viewing experience.

Solution to Problem

In order to achieve the above object, an information processingapparatus according to an embodiment of the present technology includesan image obtaining unit and a display control unit.

The image obtaining unit obtains a plurality of first divided imagesobtained by dividing a first image showing a first location along asecond direction substantially perpendicular to a first direction, and aplurality of second divided images obtained by dividing a second imageshowing a second location along the second direction.

The display control unit arranges and simultaneously displays theplurality of first divided images and the plurality of second dividedimages along the first direction on a display device of a user at athird location.

In such an information processing apparatus, the plurality of firstdivided images obtained by dividing the first image and the plurality ofsecond divided images obtained by dividing the second image aresimultaneously displayed side by side on the display device. This makesit possible to provide a high-quality viewing experience.

The display control unit may change a relative relationship ofvisibility between the plurality of first divided images and theplurality of second divided images in accordance with a change insituation of at least one of the first location, the second location, orthe third location.

The change in situation may include at least one of a change in motionof an object or a change in sound emitted by the object at at least oneof the first location, the second location, or the third location.

The object may include a person at the first location or the secondlocation.

The change in situation may include a call from the person to the userof the display device as the change in sound emitted by the object.

The object may include the user of the display device.

The change in situation may include a change in posture of the user ofthe display device.

The first direction may be a horizontal direction. In this case, thechange in posture of the user of the display device may be a change inposture in the horizontal direction.

The display control unit may control the display device such thatpositions of the plurality of first divided images and the plurality ofsecond divided images are different from each other in a depth directionwhen viewed from the user. In this case, the change in situation mayinclude a change in convergence distance of the user of the displaydevice.

The display control unit may control the display device such that aplurality of divided images, which corresponds to the convergencedistance of the user, is arranged closer to the user than anotherplurality of divided images, which does not correspond to theconvergence distance of the user, between the plurality of first dividedimages and the plurality of second divided images.

The display control unit may make the visibility of images of a locationwhere the change in situation has occurred higher than the visibility ofother images between the plurality of first divided images and theplurality of second divided images.

The information processing apparatus may further include a sound outputcontrol unit that controls a sound output device of the user to change arelative relationship between a first sound output corresponding to thefirst image and a second sound output corresponding to the second imagein accordance with a change in the relative relationship of visibility.

The sound output control unit may make a sound output corresponding to aplurality of divided images having higher visibility larger than a soundoutput corresponding to another plurality of divided images between theplurality of first divided images and the plurality of second dividedimages.

The display control unit may cause the display device to display animage in which a sound output corresponding to a plurality of dividedimages having lower visibility between the plurality of first dividedimages and the plurality of second divided images is converted intotext.

The information processing apparatus may further include a communicationcontrol unit that transmits an image to a first display device at thefirst location and a second display device at the second location. Inthis case, the image obtaining unit may obtain an image showing thethird location and including an image of the user of the display deviceobtained by an imaging device. In this case, the communication controlunit may transmit the image showing the third location to the firstdisplay device and the second display device simultaneously and in realtime.

An information processing method according to an embodiment of thepresent technology is an information processing method executed by acomputer system, the method including: obtaining a plurality of firstdivided images obtained by dividing a first image showing a firstlocation along a second direction substantially perpendicular to a firstdirection, and a plurality of second divided images obtained by dividinga second image showing a second location along the second direction; andarranging and simultaneously displaying the plurality of first dividedimages and the plurality of second divided images along the firstdirection on a display device of a user at a third location.

A program according to an embodiment of the present technology causes acomputer system to execute the steps of: obtaining a plurality of firstdivided images obtained by dividing a first image showing a firstlocation along a second direction substantially perpendicular to a firstdirection, and a plurality of second divided images obtained by dividinga second image showing a second location along the second direction; andarranging and simultaneously displaying the plurality of first dividedimages and the plurality of second divided images along the firstdirection on a display device of a user at a third location.

Advantageous Effects of Invention

As described above, according to the present technology, it is possibleto provide a high-quality viewing experience. Note that the effectsdescribed here are not necessarily limitative, and any of the effectsdescribed in the present disclosure may be provided.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram showing an example of a functionalconfiguration of an intercommunication system according to an embodimentof the present technology.

FIG. 2 is a diagram showing an example of an image of another locationcaptured by a camera.

FIG. 3 is a schematic diagram showing a generation example of acomposite image.

FIG. 4 is a schematic diagram showing a generation example of acomposite image.

FIG. 5 is a schematic diagram showing a generation example of acomposite image.

FIG. 6 is a schematic diagram showing a generation example of acomposite image.

FIG. 7 is a schematic diagram showing a generation example of acomposite image.

FIG. 8 is a schematic diagram for describing an example of a multiplesimultaneous representation.

FIG. 9 is a schematic diagram for describing an example of a multiplesimultaneous representation.

FIG. 10 is a schematic diagram for describing an example of a multiplesimultaneous representation.

FIG. 11 is a schematic diagram for describing an example of a multiplesimultaneous representation.

FIG. 12 is a schematic diagram for describing an example of a multiplesimultaneous representation.

FIG. 13 is a flowchart showing control processing of emphasizing aspecific location on the basis of a change in situation of a location.

FIG. 14 is a flowchart showing control processing of emphasizing aspecific location in response to a call from the other party.

FIG. 15 is a flowchart showing control processing of emphasizing aspecific location on the basis of a registered or emphasized word.

FIG. 16 is a flowchart showing control processing of emphasizing alocation corresponding to a search.

FIG. 17 is a schematic diagram schematically showing an example of adisplay method for preventing blocking.

FIG. 18 is a schematic diagram schematically showing an example of adisplay method for preventing blocking.

FIG. 19 is a schematic diagram schematically showing another example ofa display method for preventing blocking.

FIG. 20 is a schematic diagram showing an example of control processingof emphasizing a specific location.

FIG. 21 is a schematic diagram schematically showing another example ofa display method for preventing blocking.

FIG. 22 is a diagram showing a configuration example of an HMD accordingto another embodiment.

FIG. 23 is a perspective view showing an external example of an HMDaccording to another embodiment.

MODE(S) FOR CARRYING OUT THE INVENTION

Embodiments according to the present technology will now be describedbelow with reference to the drawings.

[Intercommunication System]

FIG. 1 is a block diagram showing an example of a functionalconfiguration of an intercommunication system 500 according to anembodiment of the present technology. The intercommunication system 500is capable of implementing telecommunication between differentlocations.

In this embodiment, a case where communication is performed betweenpersons on respective 7th, 8th, and 9th floors of a building will beexemplified. Further, a case where a communication control apparatus 20functioning as an embodiment of an information processing apparatusaccording to the present technology is disposed on the 7th floor will beexemplified. Hereinafter, the present technology will be described bytaking a person on the 7th floor as a user and persons on the 8th and9th floors as communication partners.

In this embodiment, the 7th floor corresponds to a third location. The8th and 9th floors correspond to first and second locations. Thespecific location is not limited, and the present technology isapplicable to any location. Further, the number of locations is notlimited, and the present technology is applicable to any number of threeor more locations.

The intercommunication system 500 includes a communication system 100built on the 7th floor and a communication system 200 built on each ofthe 8th and 9th floors. Note that FIG. 1 shows one communication system200, but the communication system 200 is built on each of the 8th and9th floors one by one.

The communication system 100 includes a camera 11, a microphone 12, adisplay 13, a speaker 14, and a communication control apparatus 20. Thecamera 11, the microphone 12, the display 13, and the speaker 14 areconnected to the communication control apparatus 20. The method ofconnecting the camera 11 and the like to the communication controlapparatus 20 is not limited, and any communication method such as awireless method, a wired method, or the like may be employed.

The camera 11 is capable of capturing an image of the 7th floor wherethe user is located. Thus, an image showing the 7th floor (image data)is obtained by the camera 11. Of course, when the user is on the 7thfloor, an image including an image of the user is captured. In thisembodiment, the camera 11 functions as an imaging device that obtains animage showing a third location including an image of the user. Thespecific configuration of the camera 11 is not limited.

Various images and GUIs are displayed on the display 13. For example, animage of the 7th floor captured by the camera 11 is displayed on thedisplay 13. Further, images of the 8th and 9th floors captured by thecameras 31 installed in the respective 8th and 9th floors are displayed.Of course, when there is a person on the 8th or 9th floor, an imageincluding an image of the person is displayed.

In this embodiment, a composite image 80 to be described later (see FIG.7) is displayed on the display 13. In addition, any image such as a textimage, an icon image, or a cursor may be displayed. In this embodiment,the display 13 corresponds to a display device of the user. The specificconfiguration of the display 13 is not limited. For example, the display13 may be configured by a television device, a 3D TV device, a headmounted display (HMD), or the like.

The microphone 12 is capable of collecting sounds generated on the 7thfloor where the user is located. For example, the microphone 12 detectsthe voice uttered by the user or surrounding sounds. The specificconfiguration of the microphone 12 is not limited.

The speaker 14 is capable of outputting various sounds. For example, thespeaker 14 is capable of outputting the voice of the user or the likeobtained by the microphone 12. Further, the speaker 14 is capable ofoutputting sounds obtained by the microphones 32 installed in the 8thand 9th floors. For example, the voices spoken by persons on the 8th and9th floors can be output from the speaker 14. The specific configurationof the speaker 14 is not limited.

The user can make a voice call with a person on another floor via themicrophone 12 and the speaker 14. The user can also listen intently tosounds generated on other floors. The user can also input variousinstructions to the communication control apparatus 20 or the like byvoice input.

In this embodiment, the person at each location is included in anobject. That is, the user on the 7th floor, the communication partnerson the 8th and 9th floors are included in the object according to thepresent technology. Note that the object is not limited to a person, andany object such as an animal, a robot, various devices, and the like isalso included in the object according to the present technology.

The communication control apparatus 20 includes, for example, hardwarenecessary for the configuration of a computer such as a CPU (processor),a ROM, a RAM, and an HDD. The information processing method according tothe present technology is executed when the CPU loads a programaccording to the present technology, which is recorded in advance in theROM or the like, into the RAM and executes the program.

For example, the communication control apparatus 20 can be implementedby any computer such as a personal computer (PC). It is needless to saythat any hardware such as a GPU, a FPGA, or ASIC may be used toimplement the communication control apparatus 20.

In this embodiment, the CPU or the like of the communication controlapparatus 20 executes a predetermined program, and thus an image inputunit 21, a sound input unit 22, an image output unit 23, a sound outputunit 24, a user state recognition unit 25, a reproduction control unit26, and a communication control unit 27 are implemented as functionalblocks. In order to implement each block, dedicated hardware such as anintegrated circuit (IC) may be used.

The program is installed on the communication control apparatus 20through, for example, various recording media. Alternatively, theinstallation of the program may be executed via the Internet or thelike.

The type or the like of the recording medium on which the programaccording to the present technology is recorded is not limited, and anycomputer readable recording medium may be used. For example, anyrecording medium that non-temporarily records data may be used.

An image captured by the camera 11 is input to the image input unit 21.In this embodiment, an image of the 7th floor where the user is locatedis input. That is, in this embodiment, an image showing the thirdlocation including the image of the user of the display device obtainedby the imaging device is input to the image input unit 21. In thisembodiment, the image input unit 21 is capable of functioning as a partof the image obtaining unit.

Note that, in the present disclosure, the image includes both a stillimage and a moving image (video). In this embodiment, a video includinga plurality of frames captured at a predetermined frame rate is capturedby the camera 11. The video data (a plurality of frames) thereof isinput to the image input unit 21.

A sound (sound data) detected by the microphone 12 is input to the soundinput unit 22.

The image output unit 23 outputs an image by controlling the displayoperation of the display 13. In this embodiment, the reproductioncontrol unit 26 outputs image data and a control signal to be displayedon the display 13 to the image output unit 23. The image output unit 23controls the display operation of the display 13 on the basis of theimage data and the control signal output from the reproduction controlunit 26.

The sound output unit 24 outputs a sound by controlling a sound outputoperation of the speaker 14. In this embodiment, the reproductioncontrol unit 26 outputs sound data and a control signal to be outputfrom the speaker 14 to the sound output unit 24. The sound output unit24 controls the sound output operation of the speaker 14 on the basis ofthe sound data and the control signal output from the reproductioncontrol unit 26.

The user state recognition unit 25 recognizes the state of the user onthe 7th floor on the basis of the image input to the image input unit 21and the sound input to the sound input unit 22. In this embodiment, theuser state recognition unit 25 recognizes the line of sight of the user,the convergence of the user, the posture of the user, the contents ofthe utterance of the user, and the like.

For example, the user state recognition unit 25 is capable ofrecognizing a change in the line of sight of the user, a change in theconvergence of the user, a change in the posture of the user such aslooking in the display 13, and the like. In addition, the user staterecognition unit 25 is capable of recognizing the contents of anutterance such as a call to a communication partner at another location.Note that the utterance of the user corresponds to a change in soundemitted by the user (change in sound). Consequently, recognizing thecontents of the utterance of the user corresponds to recognizing thechange in sound emitted by the user.

In this embodiment, a change in the line of sight of the user, a changein the convergence of the user, a change in the posture of the user, andan utterance of the user (change in sound emitted by the user) areincluded in a change in situation of the 7th floor (third location). Forexample, a call of a communication partner on another floor correspondsto a change in sound emitted by the user and is included in a change insituation of the 7th floor.

The specific method for recognizing the state of the user is notlimited, and any technique may be used. For example, anymachine-learning algorithm using a deep neural network (DNN) or the likemay be used. For example, it is possible to improve the recognitionaccuracy of the state of the user by using artificial intelligence (AI)or the like for performing deep learning.

The communication control unit 27 controls communication with thecommunication control apparatuses 40 disposed on the other floors(locations on the 8th and 9th floors). Note that the method ofcommunicably connecting the communication control apparatuses 20 (40) ofthe respective floors is not limited. For example, any network such as awide area network (WAN) or a local area network (LAN) is used. Thecommunication control unit 27 is capable of transmitting and receivingvarious types of information (data) by controlling communication devicessuch as a module for establishing communication and a router.

In this embodiment, the communication control unit 27 is capable oftransmitting the image input to the image input unit 21 and the soundinput to the sound input unit 22 to the communication control unit 49 onanother floor. Further, the communication control unit 27 is capable oftransmitting data of a recognition result of the user state recognitionunit 25. Further, the communication control unit 27 receives varioustypes of information (data) output from the communication control unit49 of another communication control apparatus 40.

In this embodiment, the communication control unit 27 is capable oftransmitting images to the display 33 on the 8th (9th) floor. Thecommunication control unit 27 is also capable of transmitting an imageshowing the 7th floor, for example, including an image of the user ofthe display 13 obtained by the camera 11, to the display 33 on the 8thfloor and to the display 33 on the 9th floor simultaneously and in realtime.

The reproduction control unit 26 controls the image display by thedisplay 13 and the sound output by the speaker 14. The reproductioncontrol unit 26 outputs the image data and the control signal to bedisplayed on the display 13 to the image output unit 23. Further, thereproduction control unit 26 outputs the sound data and the controlsignal to be output from the speaker 14 to the sound output unit 24.

The reproduction control unit 26 is capable of controlling the imagedisplay and the sound output on the basis of the image input to theimage input unit 21, the sound input to the sound input unit, therecognition result of the user state recognition unit 25, and varioustypes of information received by the communication control unit 27.

In this embodiment, a composite image 80 to be described later (see FIG.7) is generated by the reproduction control unit 26. The display of thecomposite image 80 is then controlled. In this embodiment, thereproduction control unit 26 functions as an image obtaining unit, adisplay control unit, and a sound output control unit.

The communication system 200 built on each of the 8th and 9th floorsincludes a camera 31, a microphone 32, a display 33, a speaker 34, andthe communication control apparatus 40. The camera 31, the microphone32, the display 33, and the speaker 34 are connected to thecommunication control apparatus 40.

The camera 31 is capable of capturing an image of the 8th (9th) floorwhere a communication partner is located. Thus, an image showing the 8th(9th) floor (image data) is obtained by the camera 31. Of course, whenthe communication partner is on the 8th (9th) floor, an image includingan image of the communication partner is captured.

Various images and GUIs are displayed on the display 33. For example, animage of each floor (including an image of a person) is displayed on thedisplay 33. In addition, any image such as a text image, an icon image,or a cursor may be displayed. In this embodiment, the display 13functions as a first device at a first location (a second display deviceat a second location).

The microphone 32 is capable of collecting sounds (voices) generated onthe 8th (9th) floor where the communication partner is located. Forexample, the microphone 32 detects the voice uttered by thecommunication partner or surrounding sounds.

The speaker 34 is capable of outputting various sounds. For example, thespeaker 34 is capable of outputting the voice uttered by a person oneach floor, a sound generated on each floor, or the like.

For example, a person on the 8th (9th) floor can make a voice call witha person on another floor via the microphone 32 and the speaker 34.Further, the person on the 8th (9th) floor can also listen intently tosounds generated on other floors. Further, the person on the 8th (9th)floor can also input various instructions to the communication controlapparatus 40 or the like by voice input.

The communication control apparatus 40 includes, for example, hardwarenecessary for the configuration of a computer such as a CPU (processor),a ROM, a RAM, and an HDD. The information processing method according tothe present technology is executed when the CPU loads a programaccording to the present technology, which is recorded in advance in theROM or the like, into the RAM and executes the program.

For example, the communication control apparatus 40 can be implementedby any computer such as a personal computer (PC). It is needless to saythat any hardware such as a GPU, a FPGA, or ASIC may be used toimplement the communication control apparatus 20.

In this embodiment, the CPU or the like of the communication controlapparatus 40 executes a predetermined program, and thus an image inputunit 41, a sound input unit 42, an image output unit 43, a sound outputunit 44, an environment recognition unit 45, an utterance analysis unit46, a person/object detection unit 47, a reproduction control unit 48,and a communication control unit 49 are implemented as functionalblocks. The method of installing the program is not limited.

Further, in this embodiment, a person/object registration DB 50 is builtin the communication control apparatus 40. The person/objectregistration DB 50 is built by, for example, a storage medium such as anHDD provided in the communication control apparatus 40. Needless to say,the present technology is not limited thereto, and the person/objectregistration DB 50 may be constructed by a storage device configuredseparately from the communication control apparatus 40.

An image captured by the camera 31 is input to the image input unit 41.In this embodiment, an image of the 8th (9th) floor where thecommunication partner is located is input. That is, in this embodiment,a first image (second image) showing the first location (secondlocation) is input to the image input unit 41.

A sound (sound data) detected by the microphone 12 is input to the soundinput unit 42.

The image output unit 43 outputs an image by controlling the displayoperation of the display 33. In this embodiment, the reproductioncontrol unit 48 outputs image data and a control signal to be displayedon the display 33 to the image output unit 43. The image output unit 43controls the display operation of the display 33 on the basis of theimage data and the control signal output from the reproduction controlunit 48.

The sound output unit 44 outputs a sound by controlling a sound outputoperation of the speaker 34. In this embodiment, the reproductioncontrol unit 48 outputs sound data and a control signal to be outputfrom the speaker 34 to the sound output unit 44. The sound output unit44 controls the sound output operation of the speaker 34 on the basis ofthe sound data and the control signal output from the reproductioncontrol unit 48.

The environment recognition unit 45 recognizes the environment of the8th (9th) floor on the basis of the image input to the image input unit41 and the sound input to the sound input unit 42. For example, theenvironment recognition unit 45 recognizes an operation of a person oran object on the 8th (9th) floor, sounds such as noise generated on thefloor, or the like. The technologies for environment recognition are notlimited, and any algorithm such as machine learning may be used, forexample.

The utterance analysis unit 46 analyzes the utterance of the person onthe basis of the sound input to the sound input unit 42. For example,the utterance analysis unit 46 is capable of recognizing the contents ofan utterance such as a call from a person on the 8th (9th) floor to aperson at another location. In addition, the utterance analysis unit 46is capable of analyzing whether the contents of the utterance of theperson include a particular keyword registered in the person/objectregistration DB 50. The technologies for the utterance analysis are notlimited, and any algorithm such as machine learning may be used, forexample.

In this embodiment, the environment recognition unit 45 and theutterance analysis unit 46 detect a change in situation of the locationon the 8th (9th) floor. For example, an environmental change of the 8th(9th) floor is detected as a change in situation of the location.Further, an utterance of a person on the 8th (9th) floor (change insound emitted by the person) is detected as a change in situation of thelocation. For example, a call of a person on the 8th (9th) floor to aperson on the 7th floor can also be detected as a change in situation ofthe location.

Further, the function of the user state recognition unit 25 in thecommunication control apparatus 20 on the 7th floor may be provided inthe communication control apparatus 40 on the 8th (9th) floor. Changesin the line of sight, convergence, posture, or the like of the person onthe 8th (9th) floor may be detected as changes in situation of thelocation. For example, a motion of looking in the display 33 of a personon the 8th (9th) floor may be detected as a change in situation of thelocation.

The person/object detection unit 47 detects a person or an objectappearing in the image on the basis of an image input to the image inputunit 41. For example, it is determined whether or not a person or anobject appearing in the image is a person or an object registered in theperson/object registration DB 50. The technologies for detecting aperson or an object appearing in the image are not limited, and anyimage analysis technology such as the matching technology may be used.Of course, machine learning or the like may be used.

Information of a predetermined person, information of an object, aparticular word, and the like are registered in the person/objectregistration DB 50. For example, in the person/object registration DB50, information such as images and names of staff, etc. of a company orthe like in which the communication system 500 is built, and images orarrangement places of fixtures are registered. In addition,predetermined words and the like used for conversations are registeredin the person/object registration DB 50.

The reproduction control unit 48 controls the image display by thedisplay 33 and the sound output by the speaker 34. The reproductioncontrol unit 48 outputs the image data and the control signal to bedisplayed on the display 33 to the image output unit 43. Further, thereproduction control unit 48 outputs the sound data and the controlsignal to be output from the speaker 34 to the sound output unit 44.

The communication control unit 49 controls communication with thecommunication control apparatus 20 disposed on the 7th floor. In thisembodiment, the communication control unit 49 is capable of transmittingthe image input to the image input unit 41 and the sound input to thesound input unit 42 to the communication control unit 49 on the 7thfloor. That is, an image of the 8th floor and an image of the 9th floor,which respectively correspond to the image showing the first locationand the image showing the second location, are transmitted from thecommunication control unit 49 to the communication control unit 27.Further, voice such as an utterance of a person on the 8th floor andvoice such as an utterance of a person on the 9th floor are transmittedfrom the communication control unit 49 to the communication control unit27.

Further, the communication control unit 49 transmits the recognitionresult by the environment recognition unit 45, the analysis result bythe utterance analysis unit 46, and the detection result by theperson/object detection unit 47 to the communication control unit 27 onthe 7th floor. In addition, any information (data) may be transmitted tothe communication control unit 27 on the 7th floor. Of course,communication is appropriately executed between the communicationcontrol unit 49 of the communication control apparatus 40 on the 8thfloor and the communication control unit 49 of the communication controlapparatus 40 on the 9th floor.

[Composite Image]

FIG. 2 is a diagram showing an example of an image of another locationcaptured by the camera 31. A of FIG. 2 is an image of the 8th floor, andB of FIG. 2 is an image of the 9th floor. On the 8th floor, a conferenceis being carried out by persons 61 and 62. On the 9th floor, apresentation of a new product 73 is being carried out by persons 71 and72.

FIGS. 3 to 7 are schematic diagrams each showing a generation example ofa composite image 80. In this embodiment, a composite image is generatedby the reproduction control unit 26 in the communication controlapparatus 20 on the 7th floor. A composite image is an image that canprovide a viewing experience as if the user on the 7th floor weresimultaneously located on each of the 8th and 9th floors serving as theother locations. The user can obtain a virtual experience as if the userwere simultaneously present on the 8th and 9th floors while being on the7th floor.

An image 60 of the 8th floor and an image 70 of the 9th floor are outputto the reproduction control unit 26 in the communication controlapparatus 20 on the 7th floor via the communication control unit 27 andthe communication control units 49.

As shown in FIG. 3, the reproduction control unit 26 divides the image60 of the 8th floor into a plurality of images along a direction (seconddirection) substantially perpendicular to a predetermined direction(first direction). The divided images are arranged in a predetermineddirection. Here, the substantially perpendicular direction may beregarded as one corresponding to separation lines (boundary lines) fordividing the image.

In this embodiment, the image 60 of the 8th floor is divided into aplurality of images along the lateral direction (Y direction) of theimage. The separation lines for dividing the image 60 of the 8th floorextend in the vertical direction (Y direction) of the image. Note thatin this embodiment the display 13 is set such that the lateral directionof the image is the horizontal direction.

The image 60 of the 8th floor is divided at equal intervals along thehorizontal direction (X direction). Therefore, the widths (size inhorizontal walking) of the plurality of divided images (hereinafter,referred to as first divided images) 65 generated by the division arethe same size.

As shown in FIG. 4, the reproduction control unit 48 divides the image70 of the 9th floor into a plurality of images along the perpendiculardirection. In this embodiment, similarly to the image 60 of the 8thfloor, the image 70 of the 9th floor is divided into a plurality ofimages along the perpendicular direction of the image.

The image 70 of the 9th floor is arranged at equal intervals in thehorizontal direction. Therefore, the widths (size in horizontal walking)of the plurality of divided images (hereinafter, referred to as seconddivided images) 75 generated by the division are the same size. Further,the width of the first divided image 65 and the width of the seconddivided image 75 are set to be equal to each other.

The widths of the first divided image 65 and the second divided image 75are not limited and may be arbitrarily set. For example, the widths ofthe first divided image 65 and the second divided image 75 are set inunits of pixels. For example, the width of a portion of a place(coordinates) of the location where an important object such as a personexists may be reduced. Needless to say, the width is not limitedthereto, and the width of the first divided image 65 and the width ofthe second divided image 75 may be set by a length of millimeters or thelike.

FIG. 5 is an image showing a state in which the odd-numbered firstdivided images 65 from the left side of the image are selected in theplurality of first divided images 65. FIG. 6 is an image showing a statein which the even-numbered second divided images 75 from the left sideof the image are selected in the plurality of second divided images 75.

FIG. 7 is a schematic diagram showing an example of the composite image80. The reproduction control unit 26 simultaneously displays theplurality of first divided images 65 shown in FIG. 5 and the pluralityof divided images 75 shown in FIG. 6 on the display 13 of the user onthe 7th floor along the horizontal direction which is a predetermineddirection.

In this embodiment, the plurality of first divided images 65 and theplurality of divided images 75 are arranged and simultaneously displayedsuch that each of the plurality of second divided images 75 issandwiched between the plurality of first divided images 65. That is,the composite image 80 is generated such that the first divided image 65and the second divided image 75 are alternately arranged. Thus, amultiple simultaneous representation that allows a simultaneousexperience of a plurality of locations, which will be described later,is achieved.

In this embodiment, the plurality of first divided images obtained bydividing the first image showing the first location along theperpendicular direction, and the plurality of second divided imagesobtained by dividing the second image showing the second location alongthe perpendicular direction are obtained by the reproduction controlunit 26 functioning as an image obtaining unit.

Subsequently, the plurality of first divided images and the plurality ofsecond divided images are arranged along the horizontal direction andsimultaneously displayed on the display device of the user at the thirdlocation by the reproduction control unit 26 functioning as a displaycontrol unit.

The specific method for generating the composite image is not limitedand may be arbitrarily set. That is, how the image 60 of the 8th flooris divided, how the plurality of first divided images 65 is selected,how the image 70 of the 9th floor is divided, how the plurality ofsecond divided images 75 is selected, and how the plurality of firstdivided images 65 and the plurality of second divided images 75 arearranged may be arbitrarily set so as to achieve a desired multiplesimultaneous representation.

For example, the division method along the predetermined direction forthe image 60 of the 8th floor and the division method along the samedirection for the image 70 of the 9th floor may be different methods.For example, the width, interval, and the like of the first dividedimage 65 may be different from the width, interval, and the like of thesecond divided image 75. Further, the first divided images 65 may havedifferent widths, or the second divided images 75 may have differentwidths. For example, the floor image 60 or the floor image 70 may not bedivided at equal intervals. Also, if there is no object of interest inthe divided image, for example, there is no person, the display may notbe performed. Further, for example, the width of each divided image maybe controlled in accordance with the number of people in the image.

The arrangement of the plurality of first divided images 65 and theplurality of divided images 75 along a predetermined direction is alsolimited to the case of alternately arranging the images, and anyarrangement may be employed.

Further, software image processing may be performed on the image data(pixel data) of the first divided images 65 and the second dividedimages 75. For example, the size, shape, color, contrast, direction, andthe like of a person or an object displayed in the divided image may beappropriately changed by image processing. Various techniques may beperformed so as to implement a desired multiple simultaneousrepresentation using the composite image 80.

[Multiple Simultaneous Representation]

Several examples of the multiple simultaneous representation implementedby the composite image 80 will be described. FIGS. 8 to 12 are schematicdiagrams for describing an example of the multiple simultaneousrepresentation.

As shown in FIG. 8, it is possible to implement the multiplesimultaneous representation in which the display of the 8th floor andthe display of the 9th floor are switched to each other in accordancewith a change in the line of sight of a user with respect to the display13.

By switching the line-of-sight direction (angle of line of sight) withrespect to the display 13, a user 90 can simultaneously obtain a virtualexperience in which the user 90 is holding a conference with the persons61 and 62 on the 8th floor and a virtual experience in which the user 90is participating in the presentation by the persons 71 and 72 on the 9thfloor.

As shown in FIG. 9, it is possible to implement the multiplesimultaneous representation in which the display position of the 8thfloor and the display position of the 9th floor are different from eachother in the depth direction when viewed from the user 90. For example,the scene of the conference on the 8th floor is displayed on the nearside when viewed from the user 90. The scene of the presentation on the9th floor is displayed on the far side (back side) in the depthdirection. It is possible for the user to simultaneously experience thescene of the 8th floor and the scene of the 9th floor.

The multiple simultaneous representation shown in FIG. 10 correspond tothe control of the display 13 such that the positions of the pluralityof first divided images 65 and the plurality of second divided images 75are made different from each other in the depth direction when viewedfrom the user 90.

For the user 90, the visibility of the plurality of first divided imagespositioned on the near side is higher. That is, the sensation of beingon the 8th floor is stronger. Of course, the composite image 80 isgenerated such that the sensation of being on the 9th floor can also beprovided at the same time. In this manner, it is also possible togenerate the composite image 80 such that visibility is differentbetween the plurality of first divided images 65 and the plurality ofsecond divided images 75.

Of course, it is also possible to generate the composite image 80 suchthat the plurality of second divided images 75 is positioned on the nearside and the plurality of first divided images 65 is positioned on thefar side. In this case, the visibility of the plurality of seconddivided images 75 is higher for the user 90. Therefore, for the user,the sensation of being on the 9th floor is stronger. In this manner, itis possible to change the relative relationship of the visibilitybetween the plurality of first divided images 65 and the plurality ofsecond divided images 75.

Note that the image control processing for displaying the compositeimage 80 is not limited. For example, the plurality of first dividedimages 65 and the plurality of second divided images 75 may be arrangedin a grid shape. Further, for example, the plurality of first dividedimages 65 and the plurality of second divided images 75 may be arrangedso as to have different depths, and may be controlled so as to overlapwith each other when viewed from the user.

As shown in FIG. 11, when the camera 31 for capturing images of thelocations on the 8th and 9th floors is a 360° camera or the like, thescene of the conference on the 8th floor and the scene of thepresentation on the 9th floor, which constitute the composite image 80,may be displayed on the display 13 as an omnidirectional image.

As shown in FIG. 12, in this embodiment, the reproduction control unit26 controls the divided images of the plurality of second divided images75 to be alternately seen between the divided images of the plurality offirst divided images 65 obtained by dividing the image of the locationof on the 8th floor.

If the image of the location is displayed around the user 90 like anomnidirectional image, the convergence distance of the user 90 is thedistance from the center with the user 90 as the center. That is, thedivided images of each location are controlled to appear on acircumference at a predetermined distance away from the user 90.Further, the interval between the divided images displayed on thecircumference is an arc corresponding to the angle of the adjacentdivided image when viewed from the user 90.

With reference to FIGS. 17 to 20, an example of a multiple simultaneousrepresentation, i.e., providing composite images, is furthersupplemented. As shown in FIGS. 17 to 20, images of a plurality ofspaces (floor images 660 and 670) are virtually arranged in the depthdirection. Images for the left and right eyes are obtained by virtuallycapturing the arranged images of the spaces from a position offsethorizontally so as to correspond to the average interval between theeyes of humans (about 6.5 cm). This may be regarded as a methodsubstantially similar to a general stereo pair image generation method.The images of the spaces arranged at different positions in the depthdirection (floor images 660 and 670) are respectively divided intodivided images 665 and 675 as will be described later. An appropriateoffset is provided between the divided image(s) 665 in the horizontaldirection. Similarly, an offset is provided between the divided images675. The offset of the divided images 665 and the offset of the dividedimages 675 are appropriately adjusted, so that the divided images 665and the divided images 675 can be arranged so as to be simultaneouslyvisible.

Note that it is determined which of the divided images 665 and 675 is animage fused by eyes 690 of the user 90 in accordance with theconvergence angle of the eyes 690. One of the divided images that is notfused appears blurred to the user 90. That is, the user 90 can switchbetween a clearly visible space and a blurred space only by being awareof viewing a short distance or a long distance.

[Communication Operation Using Composite Image]

The communication operation using the composite image will be described.In this embodiment, the reproduction control unit 26 (48) controls thecomposite image 80 displayed on the display 13 of the 7th floor inaccordance with a change in situation of at least one of the 7th, 8th,or 9th floor. A multiple simultaneous representation implemented by thecomposite image 80 is then appropriately changed. Note that one changein situation of at least one of the 7th, 8th, or 9th floor correspondsto a change in situation of at least one of the first location, thesecond location, or the third location.

For example, the reproduction control unit changes the relativerelationship of visibility between the plurality of first divided images65 and the plurality of second divided images 75 in the composite image80. This makes it possible to control the intensity of the sensation ofbeing on the 8th floor and the intensity of the sensation of being onthe 9th floor.

That is, the visibility of the plurality of first divided images is madehigher than the visibility of the plurality of second divided images,and thus it is possible to emphasize the sensation of being on the 8thfloor. On the other hand, the visibility of the plurality of seconddivided images is made higher than the visibility of the plurality offirst divided images, and thus it is possible to emphasize the sensationof being on the 9th floor.

In this embodiment, the relative relationship of visibility between theplurality of first divided images 65 and the plurality of second dividedimages 75 is changed on the basis of a change in motion and an utterance(change in sound) of the user 90 on the 7th floor. In addition, therelative relationship of visibility between the plurality of firstdivided images 65 and the plurality of second divided images 75 ischanged on the basis of a change in motion and an utterance (change insound) of the communication partners on the 8th and 9th floors.

For example, in response to a call from a communication partner on the8th floor to the user 90, the reproduction control unit 26 increases thevisibility of the plurality of first divided images 65 to be higher thanthe visibility of the plurality of second divided images 75, which arethe other images.

For example, as illustrated in FIG. 9, it is assumed that the compositeimage 80 is generated such that the positions of the plurality of firstdivided images 65 and the plurality of second divided images 75 aredifferent from each other in the depth direction when viewed from theuser 90. In this case, the composite image 80 is controlled such thatthe plurality of first divided images 65 is arranged closer to the user90 than the plurality of second divided images 75. That is, thecomposite image 80 is controlled so as to provide a multiplesimultaneous representation in which the 8th floor on which thecommunication partner who has called is located is emphasized.

Of course, if a communication partner on the 9th floor calls the user90, the visibility of the plurality of second divided images 75 is madehigher than that of the plurality of first divided images 65.

The relative relationship of visibility between the plurality of firstdivided images 65 and the plurality of second divided images 75 may bechanged in accordance with the change in posture of the user 90. Forexample, it is assumed that the user 90 looks in the plurality of firstdivided images 65 or the plurality of second divided images 75 in thecomposite image 80. The reproduction control unit 26 makes thevisibility of the plurality of divided images, which corresponds to thefloor of the images looked in, higher than the visibility of the otherdivided images in accordance with the looking-in motion of the user.

As shown in FIG. 7 and the like, in this embodiment, the plurality offirst divided images 65 and the plurality of second divided images 75are arranged along the horizontal direction and simultaneouslydisplayed. Therefore, it is possible to determine the presence orabsence of the looking-in motion of the user on the basis of the changein the posture of the user in the horizontal direction. Therefore, inthis embodiment, the relative relationship of visibility between theplurality of first divided images 65 and the plurality of second dividedimages 75 is appropriately changed on the basis of the change in theposture of the user in the horizontal direction.

Further, as illustrated in FIG. 10, it is assumed that the compositeimage 80 is generated such that the positions of the plurality of firstdivided images 65 and the plurality of second divided images 75 aredifferent from each other in the depth direction when viewed from theuser 90. In this case, the relative relationship of visibility betweenthe plurality of first divided images 65 and the plurality of seconddivided images 75 is changed on the basis of the change in theconvergence distance of the user.

The convergence distance is a distance between a gaze point at which theuser 90 is gazing and the user 90. For example, if the user 90 is gazingat the display 13, the convergence distance is substantially equal tothe distance between the user 90 and the display 13.

The reproduction control unit 26 controls the display 13 such that aplurality of divided images corresponding to the convergence distance ofthe user 90 in the plurality of first divided images 65 and theplurality of second divided images 75 is arranged closer to the user 90than the other plurality of divided images not corresponding to theconvergence distance of the user 90 in the plurality of first dividedimages 65 and the plurality of second divided images 75. That is, therelative relationship of visibility between the plurality of firstdivided images 65 and the plurality of second divided images 75 isappropriately changed such that the display of the floor correspondingto the convergence distance of the user 90 is emphasized.

For example, it is assumed that the composite image 80 is generated suchthat the scene of the meeting on the 8th floor is positioned on the nearside, and the scene of the presentation on the 9th floor is positionedon the back side. On the basis of the change in the convergence distanceof the user 90, the reproduction control unit 26 increases thevisibility of the image corresponding to the convergence distance, thatis, the image in which the gaze point of the user 90 is located. Forexample, if the user 90 is viewing the scene of the presentation on the9th floor, the composite image 80 is controlled such that the 9th flooris arranged close to the user.

Further, it is also possible to perform processing such that thevisibility of the images at a location where a change in situation hasoccurred among the plurality of first divided images 65 and theplurality of second divided images 75 is made higher than the visibilityof the other images. This makes it possible to quickly confirm theimages of the floor in which the change in situation has occurred. Inaddition, the composite image 80 may be controlled in accordance withvarious change in situations of each floor of the 7th, 8th, or 9thfloor. Any control method may be adopted as the control of the compositeimage corresponding to the change in situation.

Hereinafter, specific processing examples of the communication operationusing the composite image 80 will be described with reference to FIGS.13 to 16.

FIG. 13 is a flowchart showing control processing of emphasizing aspecific location on the basis of a change in situation of the location.

As shown in FIG. 7, when the locations of the 8th and 9th floors arebeing simultaneously displayed on the display 13 (Step 101), the userstate recognition unit 25 and the environment recognition unit 45determine whether or not there is a change in situation of a location atat least one of the location of the 8th floor, the location of the 9thfloor, or the location where the user is present (Step 102).

The change in situation of the location includes at least one of achange in motion of a person or a change in sound emitted by a person atat least one of the location of the 8th floor, the location of the 9thfloor, or the location where the user is present. Further, the change insituation of the location also includes a change in convergence distanceof the user of the display 13. Examples of the change in situation ofthe location include the following: the user takes a particular postureand position such as looking in; the line of sight of the user stays ata particular location longer than a certain time period; and theconvergence of the user stays at a certain depth position longer than acertain time period.

In addition, the following cases are exemplified: the fact that personsat the locations of the 8th and 9th floors perform specific actions orthe like is obtained by the image input unit 41; and many sounds inanimated conversations at the locations of the 8th and 9th floors aredetected by the sound input unit 42. For example, when the user wants toknow whether or not the construction work of the locations of the 8thand 9th floors is completed, the environment recognition unit 45recognizes whether or not the construction sound (noise) of the locationis present as the environment of the location.

If it is determined that there is a change in situation from thelocation of the 8th floor, the location of the 9th floor, and thelocation where the user is present (YES in Step 102), the reproductioncontrol unit 26 activates the location where the change in situation isrecognized to be a priority space (Step 103).

The activation includes at least one of the control of increasing thevisibility of the priority space among a plurality of locations of theother party (8th and 9th floors) compared to the other locations, or thecontrol of increasing the volume of a sound corresponding to thepriority space. That is, the activation is the control of emphasizing aparticular location.

As the examples of increasing the visibility, the reproduction controlunit 26 displays the image of the priority space in the composite image80 displayed on the display 13 on the near side, displays the image in adense color, and increases the brightness. In addition, locations otherthan the priority space may be hidden.

As the examples of the sound control, the sound output unit 24 controlsthe speaker 14 to change the relative relationship between the soundoutput of the 8th floor corresponding to the image of the 8th floor andthe sound output of the 9th floor corresponding to the image of the 9thfloor in accordance with the change in the relative relationship ofvisibility. That is, the sound output unit 24 outputs a large soundcorresponding to the priority space to the speaker 14.

In addition, the sound output unit 24 makes the sound outputcorresponding to one of the divided image 65 of the 8th floor and thedivided image 75 of the 9th floor, which has high visibility, largerthan the sound output corresponding to another one of the divided image65 of the 8th floor and the divided image 75 of the 9th floor. Inaddition, a small sound corresponding to another location may be output,or a sound corresponding to another location may not be output.

The control of reducing the visibility of other locations other than thepriority space, and the control for other locations, such as control ofreducing the volume of a sound corresponding to other locations, tofacilitate the user's recognition of the priority space can also bereferred to as deactivation. Further, the deactivation includes stoppingthe output of the image and sound on the user side to the display 33 andthe speaker 34 on the other party side. For example, when the 8th floorbecomes the priority space, the display 33 and the speaker 34 of the 9thfloor, which is another location, are in a state where the state of theuser side cannot be confirmed.

Note that the method of controlling the activation and deactivation isnot limited. For example, as the deactivation for an image of anotherlocation, images of a plurality of other locations may be controlled ina two-dimensional layer in which the images are compressed andsuperimposed at the same position in depth.

The reproduction control unit 26 deactivates the other locations at thesame time that the priority space is activated (Step 104). In addition,a notification indicating that the location of the other party has beenchanged into the priority space from the user is presented to the otherparty in the priority space in which a change in situation occurred(Step 105). For example, the user's face or a text indicating that thelocation of the other party has been changed into the priority space isdisplayed on the display 33 of the other party. Further, for example, anotification sound or the like may be output to the speaker 34 of theother party.

FIG. 14 is a flowchart showing control processing of emphasizing aspecific location in response to a call from the other party side.

It is assumed that a person on the 8th floor or the 9th floor calls whenthe user is simultaneously experiencing the 8th and 9th floors as theother locations of the other party side (Step 201). Specifically, themicrophone 32 of the other party side obtains voice of a person on theother party side. The utterance analysis unit 46 analyzes whether or notthe obtained voice is a call to the user. The utterance analysis unit 46supplies the analysis result to the reproduction control unit 26 via thecommunication control unit 49. At this time, a notification to informthe user of the location from which communication has been requested maybe output to the display 13 or the speaker 14.

In this case, the user state recognition unit 25 determines whether ornot the user permits communication with the other party in response tothe call from the other party (Step 202). For example, the user permitscommunication with the location of the other party from which the callhas been made by pressing a button that permits communication,performing a specific motion such as nodding, or making an utterance topermit communication, for example (YES in Step 202).

Similarly, when communication is not permitted, the user can rejectcommunication with the location of the other party from which the callhas been made by pressing a button that does not permit communication,performing a specific motion such as swinging the head horizontally, ormaking an utterance to reject communication, for example (NO in Step202). At this time, a notification indicating that communication hasbeen rejected may be output to the display 33 of the other party.

Note that the action to permit communication, the action to rejectcommunication, and the calling action of the other party are notlimited. For example, even if a particular action such as eye contact isperformed by a person at the location of the other party, a notificationto inform the user of the location from which communication has beenrequested may be output.

If the user permits communication with the location of the other party(YES in Step 202), the reproduction control unit 26 sets the locationfrom which the call has been made as a priority space, and actives thelocation (Step 203). Further, the reproduction control unit 26deactivates locations other than the priority space (Step 204).

For example, in the activation and deactivation when communication isperformed, images and sounds of other locations are not output to thedisplay 13 and speaker 14. That is, only the images and sounds of thepriority space are output to the display 13 and the speaker 14.

If the user does not want a person at another location to know thecontents of the conversation or the like when the user communicates witha specific location of the other party, the images and sounds on theuser side may be stopped from being output to the display 33 and thespeaker 34 of the other party.

Specifically, the signal of deactivation from the reproduction controlunit 26 is supplied to the reproduction control unit 48 of anotherlocation. As a result, the display 33 and the speaker 34 stop outputtingthe scene of the conversation between the user and the person in thepriority space.

FIG. 15 is a flow chart showing control processing of emphasizing aparticular location on the basis of a registered or emphasized word.

The sound input unit 42 obtains, from the microphone 32, the contents ofthe utterance being performed on the 8th floor or the 9th floor as thelocations of the other party (Step 301). The utterance analysis unit 46determines whether the contents of the utterance obtained by the soundinput unit 42 include a word registered in the person/objectregistration DB 50 or an emphasized word (Step 302).

The emphasized word is, for example, a word whose intonation emphasizedmore than usual is detected in the contents of the utterance of theother party, a frequently appearing word that is detected many times inthe utterance, or the like.

If a word registered in the person/object registration DB 50 or anemphasized word is detected (YES in Step 302), the location in which theword is detected is activated as a priority space (Step 303). Further,the reproduction control unit 26 deactivates locations other than thepriority space (Step 304).

For example, in the activation and deactivation when a word is detected,images of other locations are not displayed on the display 13, and theutterances from other locations are displayed as text on the display 13.

That is, the reproduction control unit 26 causes the display 13 todisplay an image in which the sound output corresponding to a pluralityof divided images having lower visibility between the plurality ofdivided images of the 8th floor and the plurality of second dividedimages of the 9th floor is converted into text. Note that the contentsof the text are not limited. For example, all utterances may bedisplayed as text, or only the registered words or emphasized words maybe displayed.

FIG. 16 is a flowchart showing control processing of emphasizing alocation corresponding to a search.

When the user searches for a specific person or object using thecommunication control apparatus 20, the user says, “Where is themanager?” to the microphone 12 (Step 401).

The user state recognition unit 25 recognizes that the user is lookingfor the manager from the user's utterance obtained by the sound inputunit 22. The person/object detection unit 47 searches the person/objectregistration DB 50 for the manager (Step 402).

If the manager is not registered in the person/object registration DB 50(NO in Step 402), the reproduction control unit 26 outputs, to the user,information indicating that the information of the manager is notregistered in the person/object registration DB 50. That is, the searchresult indicating that the search is not possible for the searchrequested by the user is fed back to the user (Step 403).

If the manager is registered in the person/object registration DB 50(YES in Step 402), the person/object detection unit 47 searches for thelocation where the manager is present in the images of the plurality oflocations obtained by the image input unit 41 (Step 404). Theperson/object detection unit 47 determines whether or not there is acorresponding person recognized as the manager in the location where thecamera 31 is installed (Step 405). For example, the person/objectdetection unit 47 recognizes a person in the image as the manager whenthe reliability indicating how much the person in the image matches theinformation of the manager registered in the person/object registrationDB 50 exceeds a threshold.

If there is a corresponding person (YES in Step 405), the reproductioncontrol unit 26 actives the location of the corresponding person as apriority space (Step 406). For example, in the activation of thecorresponding location, the visibility of the image of the priorityspace is made larger than the visibility of an image of anotherlocation, and the corresponding person is enhanced and displayed on thedisplay 13. That is, the reproduction control unit 26 increases thevisibility of the location where the corresponding person is present,and at the same time, displays the corresponding person in a large size,or attaches a predetermined marker or the like thereto, to make thecorresponding person easy to be recognized by the user.

If there is no corresponding person (NO in Step 405), the reproductioncontrol unit 26 outputs, to the user, information indicating that theinformation of the manager is not registered in the person/objectregistration DB 50 (Step 403).

As described above, in the intercommunication system 500 according tothis embodiment, the divided images 65 of the 8th floor, which areobtained by dividing the image of the 8th floor showing the location ofthe 8th floor along the perpendicular direction, and the divided images75 of the 9th floor, which are obtained by dividing the image of the 9thfloor showing the location of the 9th floor along the perpendiculardirection, are obtained. The divided images 65 of the 8th floor and thedivided images 75 of the 9th floor are simultaneously displayed side byside along the horizontal direction on the display 13 at the locationwhere the user 90 is present. This makes it possible to provide ahigh-quality viewing experience.

In virtual reality (VR), augmented reality (AR), an omnidirectionalimage as a live-action image, or the like, the user can experiencemovement to a virtual location or another location. However, it has beendifficult for a user to experience the simultaneous presence in multiplelocations, even though the user can experience movement to a singlelocation.

In this regard, in the present technology, the divided images of aplurality of locations are displayed in a striped pattern. The displayedimages or sounds corresponding to the locations are controlled on thebasis of the posture, line of sight, or the like of the user, so thatthe location with which communication can be interactively performed isswitched. This makes it possible to provide a high-quality viewingexperience.

Since the images and sounds of the plurality of locations are output atthe same time, the user can simultaneously feel the presence of theplurality of locations. Further, the communication with the plurality oflocations is emphasized and displayed under various conditions, thusmaking it easier to know the locations that the user wants to watch.

Other Embodiments

The present technology is not limited to the embodiments describedabove, and can achieve various other embodiments.

In the above embodiments, the sounds from other locations are displayedas text on the display 13. The present technology is not limited tothis. The utterance of the priority space may be displayed as text onthe display 13. That is, the user can recognize the utterancecorresponding to the priority space as sound and text.

In the above embodiments, the images of a plurality of locations of theother party are simultaneously displayed on the display 13 on the userside. The present technology is not limited to this. When there is alocation where there is no person in a plurality of locations of theother party, only the location where there is a person may be displayedon the display 13. Further, it may also be possible to make a setting todisplay a specific location at all times. Furthermore, a location otherthan a specific location may be displayed on the basis of whether or nota person enters or leaves or the presence/absence of a person. Forexample, an image of the location of the 8th floor is constantlydisplayed on the display 13, and an image of a location of the 9th flooris displayed only when there is a person.

In the above embodiments, the location where the person retrieved by theuser is located is activated. The present technology is not limited tothis. If the user retrieves a specific person, the location of thespecific person may be tracked and activated. For example, when aspecific person on the 8th floor moves to the 9th floor, thereproduction control unit 26 outputs, to the display 13, an image inwhich the location of the 8th floor is deactivated and the location ofthe 9th floor is activated.

In the above embodiments, the location of the user side and the locationof the other party are displayed on the display 13. The presenttechnology is not limited to this. An image of a location on his/her ownside (a person on the user side or the other party side) may bedisplayed on the display 13. For example, the image of the user side maybe displayed by picture-in-picture (PIP) on the display 13 on the userside. Further, when the user or a person on the other party side uses adevice capable of a virtual experience, such as an HMD, the shootablerange of the camera may be displayed as a virtual line at the virtuallocation.

Further, in the above embodiments, sounds of the location on the userside and the location on the other party side are output to the speaker14. The present technology is not limited to this. Only the soundsgenerated from the shootable range of the camera 11 may be output.Conversely, only the range in which sounds are generated within alocation may be displayed on the display 33. Further, for example, whenthe user or a person on the other party side moves, the output of thesound may be corrected on the basis of the distance or the like from thecamera 11 (31).

In the above embodiments, the user and the person at the location on theuser side and at the location on the other party side are displayed onthe display 13. The present technology is not limited to this. Anyavatars may be displayed instead of the user and the person in theactual image. Alternatively, a past image or the like of the user andthe person may be displayed instead of avatars.

In the above embodiments, a search for a person is performed. Thepresent technology is not limited to this. A search for a cart or thelike used in a company may be performed. For example, an image of thecart may be registered in the person/object registration DB 50, or imagerecognition may be performed by tagging the cart.

In the above embodiments, the priority space is activated on the basisof a change in situation or the like. The present technology is notlimited to this. For example, if a sound corresponding to a location ismoved to the left or right and the line of sight of the user follows themovement, the location where the sound is generated may be activated asthe priority space.

In the above embodiments, the camera 11 obtains an image of thelocation, and the reproduction control unit 26 divides the image intostrips. The present technology is not limited to this, and only a groupof divided images to be displayed on the display 13 may be obtained.

FIG. 17 is a schematic diagram schematically showing a composite imageaccording to another embodiment. A of FIG. 17 is a schematic diagramschematically showing a state in which the user views the images of the8th and 9th floors at differ depths. B of FIG. 17 is a schematic diagramschematically showing a display method for the user to view the imageslike a composite image. In the following description, description ofconfigurations and operations similar to those of the intercommunicationsystem 500 described in the above embodiments will be omitted orsimplified.

As shown in A of FIG. 17, the eyes of the user 690 are viewing acomposite image, i.e., an image 650 of the 8th floor and an image 670 ofthe 9th floor within a field of view 691 of the user 690. That is, theuser 690 is viewing a composite image in which the depth directions ofthe 8th and 9th floors are different. In this embodiment, only a part ofthe field of view 691 is shown in the figure to indicate the position atwhich the user wants to gaze.

Consider the case where the objects displayed in the image 660 of the8th floor and the image 670 of the 9th floor are dense when thecomposite image is displayed. As shown in B of FIG. 17, the reproductioncontrol unit 26 changes the image 660 of the 8th floor and the image 670of the 9th floor into a strip shape and outputs them to the display 13such that divided images 665 of the 8th floor and divided images 675 ofthe 9th floor do not overlap each other. As a result, the user 690 canview a composite image that allows the user to simultaneously experiencethe locations of the 8th and 9th floors.

FIG. 18 is a schematic diagram schematically showing an example of adisplay method for preventing blocking.

As shown in FIG. 18, the user 690 is gazing at the divided images 675 ofthe 9th floor. That is, the distance in the depth direction from theeyes of the user 690 to the divided images 675 of the 9th floor matchesthe convergence distance of the eyes of the user 690. In this case,blocking may occur due to the divided images 665 of the 8th floor on thenear side. The blocking means that part of the divided image blocks thefield of view 691 of the user 690, and thus part of the divided image ofthe floor at which the user 690 is gazing is not visible.

As a method of preventing the blocking, the reproduction control unit 26controls the width of the divided image 665 of the 8th floor to benarrow. Further, the reproduction control unit 26 controls the width ofthe divided image 675 of the 9th floor to be wide. That is, thereproduction control unit 26 performs control to increase the width ofthe divided image of the location at which the user is gazing, andreduces the width of the divided image of the location in front of thelocation at which the user is gazing. As a result, it is possible toprevent the field of view 691 from being blocked.

In addition, when the divided image of the location on the near side isnarrowed, the reproduction control unit 26 controls the width of thedivided image by deleting a position where there is no important personor object in the image of the location on the near side, or compressingthe entire aspect ratio, for example.

FIG. 19 is a schematic diagram schematically showing another example ofa display method for preventing blocking. A of FIG. 19 is a schematicdiagram schematically showing a state in which the divided image 675 ofthe 9th floor is blocked when viewed from the user 690. B of FIG. 19 isa schematic diagram schematically showing a state in which the eyes(viewpoint) of the user 690 are shifted.

As shown in A of FIG. 19, a field of view 692 of the user 690 is blockedby part of the divided image 665 of the 8th floor. In this case, theeyes of the user 690 fail to view part of the divided image 675 of the9th floor. As a method of preventing blocking without narrowing thewidths of the divided images in the location on the near side, as shownin B of FIG. 19, the eyes of the user 690 indicated by dashed lines arepositions in A of FIG. 19. The reproduction control unit 26 controls thedisplay at a position where the divided image 675 of the 9th floorbecomes visible even if it is blocked when the user 690 moves theviewpoint. That is, control may be performed such that the image becomesvisible when the user 690 changes the posture.

FIG. 20 is a schematic diagram showing an example of control processingof emphasizing a specific location. In the embodiments described above,images of other locations are deactivated to enhance the visibility ofthe location. The present technology is not limited to this, and thevisibility may be enhanced by changing the direction of the location atwhich the user wants to gaze.

As shown in FIG. 20, the reproduction control unit 26 controls thedirection (angle) of the divided image 665 of the 8th floor, thevisibility of which is desired to be increased, to be opposed to thedirection of the field of view 691 of the user 690. Further, thereproduction control unit 26 controls the divided image 675 of the 9thfloor to be in the direction perpendicular to the direction of thedivided image 665 of the 8th floor.

In the case of the control for changing the direction of the location,the user state recognition unit 25 recognizes the movement of the head,the gaze point, the convergence, and the like of the user. Thereproduction control unit 26 controls the angle of the image of thelocation, on which the user is focusing, on the basis of the recognizedstate of the user.

FIG. 21 is a schematic diagram schematically showing another example ofa display method for preventing blocking. A of FIG. 21 is a schematicview in which the background of the image showing the location of the8th floor is transparent. B of FIG. 21 is a schematic view in which thebackground of the image showing the location of the 9th floor istransparent.

An image 760 of the 8th floor and an image 770 of the 9th floor areimages in which the background of the scene of the conference on the 8thfloor and the background of the scene of the presentation on the 9thfloor of this embodiment are transparent. That is, the reproductioncontrol unit 26 controls image processing of extracting only importantobjects, such as a person at the location, a person of interest, anobject with which a person of interest is working, or a selected object,by image recognition or the like, and of making other backgrounds or thelike transparent.

The reproduction control unit 26 obtains divided images obtained bydividing an image 760 of the 8th floor and an image 770 of the 9th flooralong the Y-axis direction. Further, the reproduction control unit 26alternately arranges the plurality of divided images of the 8th floorand the plurality of divided images of the 9th floor along the X-axisdirection and simultaneously displays them. That is, the reproductioncontrol unit 26 performs control, on the image 760 of the 8th floor andthe image 770 of the 9th floor, similar to the image control shown inFIGS. 3 to 7, so that a composite image is generated.

The blocking may be prevented by displaying the images of the pluralityof locations, on which the above image processing has been performed, ina superimposed manner with depths. Further, for example, the positionsmay be shifted such that the overlap of persons or objects in the depthdirection is minimized.

Note that a method of preventing blocking is not limited. For example,if the number of locations is large, the depth is provided to the imageat each location to increase the distance of depth. In this case, sincethe blocking is likely to occur, the reproduction control unit 26 mayoutput the image of the location at a close position within a range inwhich the depth distance between the images of the locations can bedetermined by the convergence.

FIG. 22 is a diagram showing an exemplary configuration of an HMD 800according to another embodiment. A of FIG. 22 is a perspective viewschematically showing the external appearance of the HMD 800, and B ofFIG. 22 is a perspective view schematically showing the disassembledstate of the HMD 800.

The HMD 800 includes a base 801, an attachment band 802, headphones 803,a display unit 804, an inward-oriented camera 805 (805 a, 805 b), anoutward-oriented camera 806, and a cover 807.

The base 801 is a member disposed in front of the left and right eyes ofthe user and is provided with a frontal support portion 808 that abutson the frontal portion of the user.

The attachment band 802 is attached to the head of the user. As shown inFIG. 22, the attachment band 802 includes a temporal band 809 and aparietal band 810. The temporal band 809 is connected to the base 801and is attached so as to surround the head of the user from the temporalportion to the occipital portion. The parietal band 810 is connected tothe temporal band 809 and is attached so as to surround the head of theuser from the temporal portion to the parietal portion.

The headphones 803 are connected to the base 801 and disposed to coverthe left and right ears of the user. The headphones 803 are providedwith left and right speakers. The positions of the headphones 803 aremanually or automatically controllable. The configuration for that isnot limited, and any configuration may be adopted.

The display unit 804 is inserted into the base 801 and disposed in frontof the user's eyes. A display is disposed inside the display unit 804.Any display device using, for example, liquid crystal orelectroluminescence (EL) may be used as the display. Further, a lenssystem (not shown) for guiding an image displayed by the display to theleft and right eyes of the user is disposed in the display unit 804.

The inward-oriented camera 805 includes a left-eye camera 805 a and aright-eye camera 805 b that are capable of capturing images of the lefteye and the right eye of the user. The left-eye camera 805 a and theright-eye camera 805 b are installed at predetermined positions of theHMD 800, specifically, at predetermined positions of the base 801. Forexample, line-of-sight information or the like relating to the line ofsight of the user can be detected on the basis of the images of the lefteye and the right eye captured by the left-eye camera 805 a and theright-eye camera 805 b.

As the left-eye camera 805 a and the right-eye camera 805 b, forexample, a digital camera including an image sensor such as acomplementary metal-oxide semiconductor (CMOS) sensor or a chargecoupled device (CCD) sensor is used. Further, for example, an infraredcamera that includes an infrared illumination such as an infrared LEDmay be used.

The outward-oriented camera 806 is disposed at the center of the cover807 toward the outside (opposite side to the user). The outward-orientedcamera 806 is capable of capturing an image of a real space on the frontside of the user. A digital camera that includes, for example, an imagesensor such as a CMOS sensor or a CCD sensor is used as theoutward-oriented camera 806.

The cover 807 is attached to the base 801 and configured to cover thedisplay unit 804. The HMD 800 having such a configuration serves as animmersive head-mounted display configured to cover the field of view ofthe user. For example, a three-dimensional virtual location is displayedby the HMD 800. When wearing the HMD 800, the user can experiencevirtual reality (VR) or the like.

FIG. 23 is a perspective view showing an exemplary appearance of an HMD900 according to another embodiment. The HMD 900 is a glasses-typedevice including a transmissive display and is also referred to as ARglasses. The HMD 900 includes a frame 901, a left-eye lens 902 a and aright-eye lens 902 b, a left-eye display 903 a and a right-eye display903 b, a left-eye camera 904 a and a right-eye camera 904 b, and anoutward-oriented camera 905.

The frame 901 has the shape of glasses and includes a rim portion 906and temple portions 907. The rim portion 906 is a portion disposed infront of the right and left eyes of the user and supports each of theleft-eye lens 902 a and the right-eye lens 902 b. The temple portions907 extend rearward from both ends of the rim portion 906 toward bothears of the user and are attached to both ears at the tips. The rimportion 906 and the temple portions 907 are formed of, for example,material such as a synthetic resin or metal.

The left-eye lens 902 a and the right-eye lens 902 b are respectivelydisposed in front of the left and right eyes of the user so as to coverat least a part of the field of view of the user. Typically, each lensis designed to correct the visual acuity of the user. Of course, thepresent technology is not limited to this, and a so-called plain-glasslens may be used.

The left-eye display 903 a and the right-eye display 903 b aretransmissive displays and are disposed so as to cover a part of regionsof the left-eye lens 902 a and the right-eye lens 902 b, respectively.That is, the left-eye lens 902 a and the right-eye lens 902 b arerespectively disposed in front of the left and right eyes of the user.

Images for the left eye and the right eye and the like are displayed onthe left-eye display 903 a and the right-eye display 903 b,respectively. The user wearing the HMD 900 can view the images displayedon the displays 903 a and 903 b at the same time as viewing a realscene. This makes it possible for the user to experience augmentedreality (AR) and the like.

For example, a virtually displayed object (virtual object) is displayedon each of the displays 903 a and 903 b. For example, CGs (ComputerGraphics) of characters or the like, photographs, letters, and the likecan be displayed as virtual objects. Of course, the virtual objects arenot limited to them, and any virtual object may be displayed.

As the left-eye display 903 a and the right-eye display 903 b, forexample, a transmissive organic electroluminescence display, a liquidcrystal display (LCD), or the like is used. In addition, the specificconfigurations of the left-eye display 903 a and the right-eye display903 b are not limited. For example, a transmissive display using anarbitrary method such as a method of projecting and displaying an imageon a transparent screen or a method of displaying an image using a prismmay be used as appropriate.

The left-eye camera 904 a and the right-eye camera 904 b areappropriately installed on the frame 901 such that images of the lefteye and the right eye of the user can be respectively captured. Forexample, line-of-sight information regarding a line of sight of the usercan be detected on the basis of the images of the left eye and the righteye that are respectively captured by the left-eye camera 904 a and theright-eye camera 904 b.

A digital camera that includes, for example, an image sensor such as aCMOS sensor or a CCD sensor is used as the left-eye camera 904 a and theright-eye camera 904 b. Further, for example, an infrared camera thatincludes an infrared illumination such as an infrared LED may be used.

The outward-oriented camera 905 is disposed in a center portion of theframe 901 (rim portion 906) to be oriented outward (toward the sideopposite to the user). The outward-oriented camera 905 is capable ofcapturing an image of a real space within the field of view of the user.Thus, the outward-oriented camera 905 is capable of generating acaptured image in which a real space appears.

In this embodiment, the outward-oriented camera 905 captures an image ofthe range on the front side as viewed from the user and the rangeincluding the display area of the transmissive display 903. That is, animage of a real space is captured so as to include a range that isvisible through the display region as viewed from the user. A digitalcamera that includes, for example, an image sensor such as a CMOS sensoror a CCD sensor is used as the outward-oriented camera 905.

The information processing method and the program according to thepresent technology may be performed, and the information processingapparatus according to the present technology may be constructed, bylinking a computer mounted on the HMD with another computer capable ofcommunicating via a network or the like or by another computer capableof communicating with the HMD.

That is, the information processing method and the program according tothe present technology can be performed not only in a computer systemformed of a single computer, but also in a computer system in which aplurality of computers operates cooperatively. Note that, in the presentdisclosure, the system refers to a set of components (such asapparatuses and modules (parts)) and it does not matter whether all ofthe components are in a single housing. Thus, a plurality of apparatusesaccommodated in separate housings and connected to each other through anetwork, and a single apparatus in which a plurality of modules isaccommodated in a single housing are both the system.

The execution of the information processing method and the programaccording to the present technology by the computer system includes, forexample, both a case in which the control of the divided images, thesetting of the priority space, the control of the blocking, and the likeare performed by a single computer; and a case in which the respectiveprocesses are performed by different computers. Further, the executionof each process by a predetermined computer includes causing anothercomputer to perform a portion of or all of the process and acquiring aresult of it.

In other words, the information processing method and the programaccording to the present technology are also applicable to aconfiguration of cloud computing in which a single function is sharedand cooperatively processed by a plurality of apparatuses through anetwork.

The respective configurations of the information processing apparatus,the user state recognition unit, the environment recognition unit, theuser recognition unit, and the like; the control flow of thecommunication system; and the like described with reference to therespective figures are merely embodiments, and any modifications may bemade thereto without departing from the spirit of the presenttechnology. In other words, for example, any other configurations oralgorithms for purpose of practicing the present technology may beadopted.

Note that the effects described in the present disclosure are merelyillustrative and not restrictive, and other effects may be obtained. Theabove description of a plurality of effects does not necessarily meanthat these effects are simultaneously exhibited. It means that at leastone of the above-described effects can be obtained depending on theconditions and the like, and of course, there is a possibility that aneffect not described in the present disclosure can be exhibited.

At least two of the features among the features of the embodimentsdescribed above can also be combined. In other words, various featuresdescribed in the respective embodiments may be combined discretionarilyregardless of the embodiments.

Note that the present technology may also take the followingconfigurations.

-   (1) An information processing apparatus, including:

an image obtaining unit that obtains

-   -   a plurality of first divided images obtained by dividing a first        image showing a first location along a second direction        substantially perpendicular to a first direction, and    -   a plurality of second divided images obtained by dividing a        second image showing a second location along the second        direction; and

a display control unit that arranges and simultaneously displays theplurality of first divided images and the plurality of second dividedimages along the first direction on a display device of a user at athird location.

-   (2) The information processing apparatus according to (1), in which

the display control unit changes a relative relationship of visibilitybetween the plurality of first divided images and the plurality ofsecond divided images in accordance with a change in situation of atleast one of the first location, the second location, or the thirdlocation.

-   (3) The information processing apparatus according to (2), in which

the change in situation includes at least one of a change in motion ofan object or a change in sound emitted by the object at at least one ofthe first location, the second location, or the third location.

-   (4) The information processing apparatus according to (3), in which

the object includes a person at the first location or the secondlocation.

-   (5) The information processing apparatus according to (4), in which

the change in situation includes a call from the person to the user ofthe display device as the change in sound emitted by the object.

-   (6) The information processing apparatus according to any one of (3)    to (5), in which

the object includes the user of the display device.

-   (7) The information processing apparatus according to (6), in which

the change in situation includes a change in posture of the user of thedisplay device.

-   (8) The information processing apparatus according to (7), in which

the first direction is a horizontal direction, and

the change in posture of the user of the display device is a change inposture in the horizontal direction.

-   (9) The information processing apparatus according to any one of (6)    to (8), in which

the display control unit controls the display device such that positionsof the plurality of first divided images and the plurality of seconddivided images are different from each other in a depth direction whenviewed from the user, and

the change in situation includes a change in convergence distance of theuser of the display device.

-   (10) The information processing apparatus according to (9), in which

the display control unit controls the display device such that aplurality of divided images, which corresponds to the convergencedistance of the user, is arranged closer to the user than anotherplurality of divided images, which does not correspond to theconvergence distance of the user, between the plurality of first dividedimages and the plurality of second divided images.

-   (11) The information processing apparatus according to any one    of (2) to (10), in which

the display control unit makes the visibility of images of a locationwhere the change in situation has occurred higher than the visibility ofother images between the plurality of first divided images and theplurality of second divided images.

-   (12) The information processing apparatus according to any one    of (2) to (11), further including

a sound output control unit that controls a sound output device of theuser to change a relative relationship between a first sound outputcorresponding to the first image and a second sound output correspondingto the second image in accordance with a change in the relativerelationship of visibility.

-   (13) The information processing apparatus according to (12), in    which

the sound output control unit makes a sound output corresponding to aplurality of divided images having higher visibility larger than a soundoutput corresponding to another plurality of divided images between theplurality of first divided images and the plurality of second dividedimages.

-   (14) The information processing apparatus according to (12) or (13),    in which

the display control unit causes the display device to display an imagein which a sound output corresponding to a plurality of divided imageshaving lower visibility between the plurality of first divided imagesand the plurality of second divided images is converted into text.

-   (15) The information processing apparatus according to any one    of (1) to (14), further including

a communication control unit that transmits an image to a first displaydevice at the first location and a second display device at the secondlocation, in which

the image obtaining unit obtains an image showing the third location andincluding an image of the user of the display device obtained by animaging device, and

the communication control unit transmits the image showing the thirdlocation to the first display device and the second display devicesimultaneously and in real time.

-   (16) An information processing method executed by a computer system,    the method including:

obtaining a plurality of first divided images obtained by dividing afirst image showing a first location along a second directionsubstantially perpendicular to a first direction, and a plurality ofsecond divided images obtained by dividing a second image showing asecond location along the second direction; and

arranging and simultaneously displaying the plurality of first dividedimages and the plurality of second divided images along the firstdirection on a display device of a user at a third location.

-   (17) A program causing a computer system to execute the steps of:

obtaining a plurality of first divided images obtained by dividing afirst image showing a first location along a second directionsubstantially perpendicular to a first direction, and a plurality ofsecond divided images obtained by dividing a second image showing asecond location along the second direction; and

arranging and simultaneously displaying the plurality of first dividedimages and the plurality of second divided images along the firstdirection on a display device of a user at a third location.

REFERENCE SIGNS LIST

-   13 display-   14 speaker-   20 communication control apparatus-   23 image output unit-   24 sound output unit-   26 reproduction control unit-   27 communication control unit-   33 display-   34 speaker-   43 image output unit-   44 sound output unit-   48 reproduction control unit-   49 communication control unit-   60 image of 8th floor-   65 plurality of first divided images-   70 image of 9th floor-   75 plurality of second divided images-   80 composite image

1. An information processing apparatus, comprising: an image obtainingunit that obtains a plurality of first divided images obtained bydividing a first image showing a first location along a second directionsubstantially perpendicular to a first direction, and a plurality ofsecond divided images obtained by dividing a second image showing asecond location along the second direction; and a display control unitthat arranges and simultaneously displays the plurality of first dividedimages and the plurality of second divided images along the firstdirection on a display device of a user at a third location.
 2. Theinformation processing apparatus according to claim 1, wherein thedisplay control unit changes a relative relationship of visibilitybetween the plurality of first divided images and the plurality ofsecond divided images in accordance with a change in situation of atleast one of the first location, the second location, or the thirdlocation.
 3. The information processing apparatus according to claim 2,wherein the change in situation includes at least one of a change inmotion of an object or a change in sound emitted by the object at atleast one of the first location, the second location, or the thirdlocation.
 4. The information processing apparatus according to claim 3,wherein the object includes a person at the first location or the secondlocation.
 5. The information processing apparatus according to claim 4,wherein the change in situation includes a call from the person to theuser of the display device as the change in sound emitted by the object.6. The information processing apparatus according to claim 3, whereinthe object includes the user of the display device.
 7. The informationprocessing apparatus according to claim 6, wherein the change insituation includes a change in posture of the user of the displaydevice.
 8. The information processing apparatus according to claim 7,wherein the first direction is a horizontal direction, and the change inposture of the user of the display device is a change in posture in thehorizontal direction.
 9. The information processing apparatus accordingto claim 6, wherein the display control unit controls the display devicesuch that positions of the plurality of first divided images and theplurality of second divided images are different from each other in adepth direction when viewed from the user, and the change in situationincludes a change in convergence distance of the user of the displaydevice.
 10. The information processing apparatus according to claim 9,wherein the display control unit controls the display device such that aplurality of divided images, which corresponds to the convergencedistance of the user, is arranged closer to the user than anotherplurality of divided images, which does not correspond to theconvergence distance of the user, between the plurality of first dividedimages and the plurality of second divided images.
 11. The informationprocessing apparatus according to claim 2, wherein the display controlunit makes the visibility of images of a location where the change insituation has occurred higher than the visibility of other imagesbetween the plurality of first divided images and the plurality ofsecond divided images.
 12. The information processing apparatusaccording to claim 2, further comprising a sound output control unitthat controls a sound output device of the user to change a relativerelationship between a first sound output corresponding to the firstimage and a second sound output corresponding to the second image inaccordance with a change in the relative relationship of visibility. 13.The information processing apparatus according to claim 12, wherein thesound output control unit makes a sound output corresponding to aplurality of divided images having higher visibility larger than a soundoutput corresponding to another plurality of divided images between theplurality of first divided images and the plurality of second dividedimages.
 14. The information processing apparatus according to claim 12,wherein the display control unit causes the display device to display animage in which a sound output corresponding to a plurality of dividedimages having lower visibility between the plurality of first dividedimages and the plurality of second divided images is converted intotext.
 15. The information processing apparatus according to claim 1,further comprising a communication control unit that transmits an imageto a first display device at the first location and a second displaydevice at the second location, wherein the image obtaining unit obtainsan image showing the third location and including an image of the userof the display device obtained by an imaging device, and thecommunication control unit transmits the image showing the thirdlocation to the first display device and the second display devicesimultaneously and in real time.
 16. An information processing methodexecuted by a computer system, the method comprising: obtaining aplurality of first divided images obtained by dividing a first imageshowing a first location along a second direction substantiallyperpendicular to a first direction, and a plurality of second dividedimages obtained by dividing a second image showing a second locationalong the second direction; and arranging and simultaneously displayingthe plurality of first divided images and the plurality of seconddivided images along the first direction on a display device of a userat a third location.
 17. A program causing a computer system to executethe steps of: obtaining a plurality of first divided images obtained bydividing a first image showing a first location along a second directionsubstantially perpendicular to a first direction, and a plurality ofsecond divided images obtained by dividing a second image showing asecond location along the second direction; and arranging andsimultaneously displaying the plurality of first divided images and theplurality of second divided images along the first direction on adisplay device of a user at a third location.